Method, apparatus and non-transitory computer readable medium

ABSTRACT

Present disclosure provides a method for determining if an event relates to an unauthorized subject, the method comprising: determining a likelihood of how the event is similar to at least one of: (i) at least one event of a list of events relating to the unauthorized subject and (ii) at least one of a list of events relating to an authorized subject, each event of the list of events comprising data identifying the unauthorized subject, the determination of likelihood being based on the data identifying the unauthorized subject; and determining the event to relate to the unauthorized subject in response to the determination of the likelihood.

TECHNICAL FIELD

The present disclosure relates to a method and an apparatus fordetermining if an event relates to an unauthorized subject.

BACKGROUND ART

Video analytic for subject identification and recognition has becomeincreasingly popular in recent years. By utilizing algorithms andprocessing hardware, a video footage can be processed to obtain data foridentifying a subject in the video footage. Face recognition is one ofthe video analytic technologies that is widely used in subjectidentification. Currently, it has been adopted in public safety solutionto assist in law enforcement's investigation, authentication method fore-commerce transaction payment and contactless identity recognition inphysical access authorization. In particular, majority of facerecognition solutions determining an event of a subject appearance basedon facial information obtained from a video footage or an image as dataidentifying the subject, match the facial information identifying thesubject against facial information identifying a known or authorizedsubject, and if both facial information are highly correlated,determining the event to relate to the known or authorized subject. Invarious embodiments, an authorized subject may be one who has thepermission to enter a premise or has previously entered the premise andis determined to be unlikely to pose a threat.

An unauthorized subject detection system can be implemented through facerecognition technology based on the concept of comparing dataidentifying an unauthorized subject with data identifying an authorizedsubject. For example, if the data identifying the unauthorized subjectdo not match with the data identifying the authorized subject, an eventmay be determined to relate to the unauthorized subject. Suchunauthorized subject detection system can be useful for detectingpotential intruder for example when the intruder or an unauthorizedsubject enters a premise.

SUMMARY OF INVENTION Technical Problem

However, some of the limitations and challenges in accuracy ofconventional video analytic technologies may get amplified and hinderthe application of such video analytic technology into unauthorizedsubject detection. In particular, conventional video analytic technologymay not be able to generate a consistent outcome based on detections ofa subject detected under varying environmental or imaging conditions.Under such limitations of accuracy, an alert is immediately generatedand provided to the user once there is a possibility of detecting anunauthorized subject. A conventional technique typically does not holdan event (an instruction/a signal) to generate an alert, or determine anevent to relate to an unauthorized subject based on the determination ofanother event relating to an authorized subject and/or an unauthorizedsubject. In short, there is no technique in conventional technologies tominimize alerts that are not supposed to be generated. Therefore, it isan object of present disclosure to substantially overcome the existingchallenges as discussed above to determine if an event relates to anunauthorized subject.

An object of the present disclosure is to provide a method and anapparatus to minimize alerts that are not supposed to be generated.

Solution to Problem

According to a first aspect of the present disclosure, there is provideda method for determining if an event relates to an unauthorized subject,the method comprising: determining a likelihood of how the event issimilar to at least one of: (i) at least one event of a list of eventsrelating to the unauthorized subject and (ii) at least one of a list ofevents relating to an authorized subject, each event of the list ofevents comprising data identifying the unauthorized subject, thedetermination of likelihood being based on the data identifying theunauthorized subject; and determining the event to relate to theunauthorized subject in response to the determination of the likelihood.

According to a second aspect of the present disclosure, there isprovided an apparatus for determining if an event relates to anunauthorized subject, the apparatus comprising: a memory incommunication with a processor, the memory storing a computer programrecorded therein, the computer program being executable by the processorto cause the apparatus at least to: determine a likelihood of how theevent is similar to at least one of: (i) at least one event of a list ofevents relating to the unauthorized subject and (ii) at least one of alist of events relating to an authorized subject, each event of the listof events comprising data identifying the unauthorized subject, thedetermination of likelihood being based on the data identifying theunauthorized subject; and determine the event to relate to theunauthorized subject in response to the determination of the likelihood.

According to yet another aspect of the present disclosure, there isprovided a system for determining if an event relates to an unauthorizedsubject, the system comprising the apparatus in the second aspect and atleast one image capturing device.

Advantageous Effects of Invention

According to the present invention, it can be achieved to provide amethod and an apparatus which overcome or at least partially alleviatethe above issues.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying figures, where like reference numerals and charactersrefer to identical or functionally similar elements throughout theseparate views and which together with the detailed description beloware incorporated in and form part of the specification, serve toillustrate various embodiments and to explain various principles andadvantages in accordance with present embodiments in which:

FIG. 1 depicts a system for determining if an event relates to anunauthorized subject based on images input according to an embodiment.

FIG. 2 shows a flow chart illustrating a method for determining if anevent relates to an unauthorized subject according to an embodiment.

FIG. 3 depicts a block diagram illustrating a system for determining ifan event relates to an unauthorized subject according to an embodiment.

FIG. 4 depicts a block diagram illustrating the system of FIG. 3according to an embodiment.

FIG. 5 depict a flow chart illustrating a process of non-identifiedsubject retrieval according to an embodiment.

FIG. 6 depicts a flow chart illustrating a process of subject identifierpropagation according to an embodiment.

FIG. 7 depicts diagrams illustrating the flow chart of FIG. 6 .

FIG. 8 depicts diagram illustrating an example subject identifierpropagation process.

FIG. 9 depicts an example system for determining if an event relates toan unauthorized subject based on an input of an image capturing device.

FIG. 10 depicts a flow chart illustrating a process of online samesubject clustering according an embodiment.

FIG. 11 depicts a flow chart illustrating a process of updating subjectidentifier repository according to an embodiment.

FIG. 12 depicts a flow chart illustrating a process of updating subjectidentifier repository according to another embodiment.

FIG. 13 depicts a schematic diagram of a computer system suitable foruse to implement the systems shown in FIGS. 3 and 4 .

DESCRIPTION OF EMBODIMENTS

(Overview)

Data identifying a subject-data identifying a subject may refer toinformation that is associated with or used to identify a subject basedon the subject's attribute or characteristic information. The attributesor characteristic information can be a physical characteristic of asubject such as height, body size, hair colour, skin colour, facialinformation, apparel, belongings, other similar characteristics orcombinations, or a behavioral characteristic of a subject such as bodymovement, position of limbs, direction of movement, moving speed, theway the subject walks, stands, moves, talks, other similarcharacteristics or combinations, or other attributes or characteristicinformation. In various embodiments below, data identifying a subjectcan be obtained from an input such as an image of an imaging capturingdevice. A portion of data identifying a subject can be used a subjectidentifier (subject ID), such that a subject can be identified by thecomputer system or similar electronic device based on the subjectidentifier. For example, a subject identifier can be generated based onfacial information. Different facial information may generate twodifferent subject identifiers such that the computer system or similarelectronic device refers them as appearances of two different subjects,whereas similar facial information may be assigned with a single subjectidentifier such that the computer system or similar electronic devicerefers them as appearances of a single subject.

Event—an event refers to a signal, an instruction or an action, which isgenerated, triggered, handled or recognized by a computer system orsimilar electronic device. An event refers to a detection of anappearance of a subject based on an input during a specific time period.Each event identified from the input during the specific time period isdetermined if it relates to an authorized subject or an unauthorizedsubject based on its data identifying the subject obtained from theinput and data identifying an authorized subject retrieved from adatabase. In various embodiments below, events that are determined torelate to an authorized subject will be aggregated in a list of eventsrelating to the authorized subject, whereas events that are determinedto relate to an unauthorized subject will be aggregated in a list ofevents relating to the unauthorized subject.

Authorized subject—an authorized subject refers to a subject of a listof subjects who have the permission to enter a premise or havepreviously entered the premise and are determined to be unlikely to posea threat. In various embodiments below, data identifying the authorizedsubject are retrieved from a database of a computer system or similarelectronic device for (i) determining if an event relates to theauthorized subject and (ii) calculating a matching score of an event.

Unauthorized subject—an unauthorized subject is a subject who is not anauthorized subject or any subject in a list of authorized subjects. Invarious embodiments below, a subject whose appearance is identifiedbased on the input during the specific time period may be viewed as anunauthorized subject, and the corresponding event identified based onthe subject appearance will be detected if it relates to an unauthorizedsubject.

Matching score—a matching score is calculated by comparing dataidentifying an unauthorized subject obtained from the received inputagainst data identifying an authorized subject retrieved from thedatabase, referring to a degree of how closely both the data identifyingthe unauthorized subject and the data identifying the authorized subjectcorrespond to each other. A matching score may be scaled from 0% to100%, where a matching score of 0% indicates that both the dataidentifying the unauthorized subject and the data identifying theauthorized subject have no correlation or no characteristic informationin common, whereas a matching score of 100% indicates that both the dataidentifying the unauthorized subject and the data identifying theauthorized subject are fully correlating and identical to each other. Inan embodiment where an event is matched against a list of authorizedsubjects and a corresponding plurality of matching scores are calculatedby comparing the data identifying the unauthorized subject against dataidentifying each of the list of authorized subjects, the highestmatching score among the plurality of matching scores will be selectedas the matching score of the event.

First matching threshold—a first matching threshold refers to anunauthorized subject matching threshold which set a maximum matchingscore for an event to be determined as being related to an unauthorizedsubject. An event with a matching score lower than the first matchingthreshold is determined to relate to the unauthorized subject andassigned with an unauthorized subject identifier corresponding to theunauthorized subject. In various embodiments below, an event with anunauthorized subject identifier (unauthorized subject ID) is required toinitiate a subject identifier propagation process and used for assigningan event with the unauthorized subject identifier.

Second matching threshold—a second matching threshold refers to anauthorized subject matching threshold which set a minimum matching scorefor an event to be determined as being related to the authorizedsubject. The second matching threshold is higher than the first matchingthreshold. An event with a matching score higher than the secondmatching threshold is determined to relate to the authorized subject andassigned by an authorized subject identifier (authorized subject ID)corresponding to the authorized subject. In various embodiments below,an event with an authorized subject identifier is required to initiate asubject identifier propagation process and used for assigning an eventwith the authorized subject identifier.

Non-identified event—a non-identified event is an event that has beenassigned with a subject ID upon detecting the appearance of the subjectbut has not been assigned with an unauthorized subject ID or anauthorized subject ID. Specifically, a non-identified event has amatching score higher than the unauthorized subject matching thresholdand lower than the authorized subject matching threshold, indicating thedata identifying the unauthorized subject based on the input hasmoderate degree of correlation with the data identifying the authorizedsubject retrieved from the database. In various embodiments below,non-identified events may be retrieved and determined if they relate toan authorized subject or unauthorized subject, and assigned withauthorized subject ID or unauthorized subject ID correspondingly througha subject identifier propagation process. A newly assigned event willthen be aggregated to the corresponding list of events relating to theauthorized subject or the unauthorized subject. In some embodiments, anon-identified event may be determined as noise and may not be assignedwith an authorized subject ID or an unauthorized subject ID if it failsto relate to the authorized subject or the unauthorized subject throughthe subject identifier propagation process.

EXEMPLARY EMBODIMENTS

Embodiments of the present disclosure will be better understood andreadily apparent to one of ordinary skill in the art from the followingwritten description, which provides examples only, and in conjunctionwith the drawings.

Some portions of the description which follows are explicitly orimplicitly presented in terms of algorithms and functional or symbolicrepresentations of operations on data within a computer memory. Thesealgorithmic descriptions and functional or symbolic representations arethe means used by those skilled in the data processing arts to conveymost effectively the substance of their work to others skilled in theart. An algorithm is here, and generally, conceived to be aself-consistent sequence of steps leading to a desired result. The stepsare those requiring physical manipulations of physical quantities, suchas electrical, magnetic or optical signals capable of being stored,transferred, combined, compared, and otherwise manipulated.

Unless specifically stated otherwise, and as apparent from thefollowing, it will be appreciated that throughout the presentspecification, discussions utilizing terms such as “scanning”,“retrieving”, “determining”, “replacing”, “generating”, “initializing”,“outputting”, “receiving”, “identifying”, “predicting” or the like,refer to the action and processes of a computer system, or similarelectronic device, that manipulates and transforms data represented asphysical quantities within the computer system into other data similarlyrepresented as physical quantities within the computer system or otherinformation storage, transmission or display devices.

The present specification also discloses apparatus for performing theoperations of the methods. Such apparatus may be specially constructedfor the required purposes, or may comprise a computer or other deviceselectively activated or reconfigured by a computer program stored inthe computer. The algorithms and display presented herein are notinherently related to any particular computer or other apparatus.Various machines may be used with programs in accordance with theteachings herein. Alternatively, the construction of more specializedapparatus to perform the required method steps may be appropriated. Thestructure of a computer will appear from the description below.

In addition, the present specification also implicitly discloses acomputer program, in that it would be apparent to the person skilled inthe art that the individual steps of the method described herein may beput into effect by computer code. The computer program is not intendedto be limited to any particular programming language and implementationthereof. It will be appreciated that a variety of programming languagesand coding thereof may be used to implement the teachings of thedisclosure contained herein. Moreover, the computer program is notintended to be limited to any particular control flow. There are manyother variants of the computer program, which can use different controlflows without departing from the spirit or scope of the invention.

Furthermore, one or more of the steps of the computer program may beperformed in parallel rather than sequentially. Such a computer programmay be stored on any computer readable medium. The computer readablemedium may include storage devices such as magnetic or optical disks,memory chips, or other storage devices suitable for interfacing with acomputer. The computer readable medium may also include a hard-wiredmedium such as exemplified in the internet system, or wireless mediumsuch as exemplified in the GSM mobile telephone system. The computerprogram when loaded and executed on such as computer effectively resultsin an apparatus that implements the steps of the preferred method.

According to various embodiments below, an image capturing device may beused to capture an image comprising a subject. The subject can beidentified as appearing in the image based on characteristic informationsuch as appearance, facial information, height, hair colour, movement orother similar characteristic information, or combination. Uponidentifying the subject based on the image input, an event comprisingdata identifying the subject such as the characteristic information isdetected. Subsequently, a matching process is carried out by matchingdata identifying the subject against data identifying each subject of alist of authorized subjects to detect if the event relates to anauthorized subject. If, based on the result of the matching processindicating that the subject does not match with any authorized subjectin the list of authorized subjects, it is then determined that the eventdoes not relate to an authorized subject. As a result, an alert may begenerated indicating a detection of the unauthorized subject isdetected.

FIG. 1 depicts a system 100 for determining if an event relates to anunauthorized subject based on images input according to an embodiment. Asubject is detected in five images 101 a-101 e under differentconditions based on facial information of the subject in each image, assuch five corresponding events comprising facial information of thesubject are detected. Each event is determined if the event relates toan authorized subject by matching the subject against a list ofauthorized subjects 102 based on facial information of the subject ofthe event. In this embodiment, the image 101 e may be captured under anormal condition where the subject facial information is clear andcomplete. In such condition, the subject identified from the image 101 ecan match with an authorized subject 102 a in the list of authorizedsubjects 102, and as a result, it is determined that the event relatesto the authorized subject 102 a, as indicated in 104 a, and an alert maynot be generated (104).

In contrast, the images 101 a, 101 d may be captured under a conditionwhere the subject facial information is partially detected. In suchcondition, the subject identified from the images 101 a, 101 d could notmatch with any authorized subject in the list of authorized subjects102, and as a result, it is determined that the events do not relate toan authorized subject, as indicated in 103 a, 103 d, and the events 103a, 103 d may be used to generate an alert (103). Similarly, the image101 b, 101 c may be detected under a condition where the subject face isunclear due to low lighting and bad quality condition. In suchcondition, the subject identified from the images 101 b, 101 c could notmatch with any authorized subject in the list of authorized subjects102, and as a result, it is determined that the events do not relate toan authorized subject, as indicated in 103 b, 103 c, and the events 103b, 103 c may be used to generate an alert (103). Such alerts are falselygenerated due to varying conditions under which the images of a samesubject were taken. Therefore, it is an object of the present disclosureto manage such events and minimize an alert that is not supposed to begenerated by determining if an event relates to an unauthorized subjectprior to generating the alert.

Various embodiments provide apparatus and methods for determining if anevent relates to an unauthorized subject. FIG. 2 shows a flow chartillustrating a method 200 for determining if an event relates to anunauthorized subject according to an embodiment. At step 202, the methodcomprises a step of determining a likelihood of how an event is similarto at least one of: (i) at least one event of a list of events relatingto the unauthorized subject and (ii) at least one of a list of eventsrelating to an authorized subject, each event of the list of eventscomprising data identifying the authorized subject, the determination oflikelihood being based on the data identifying the unauthorized subject.At step 204, the method comprises a step of determining the event torelate to the unauthorized subject in response to the determination ofthe likelihood.

According to an embodiment, at step 202, the method may comprise a stepof receiving an input, the input being at least one image captured by atleast one image capturing device during a specific time period, whereinthe determination of the event and each event of the list of eventscomprising the data identifying the unauthorized subject is based on thereceived input. In particular, an event is detected when an appearanceof an unauthorized subject is identified from the input during thespecific time period, the event comprising data identifying theunauthorized subject obtained from the received input. Similarly, morethan one events may be detected when more than one appearances of theunauthorized subject are determined based on the received input duringthe specific time period, each of the more than one event comprisingrespective data identifying the unauthorized subject obtained from thereceived input.

According to an embodiment, at step 202, the method may further comprisea step of calculating a matching score of each event of the list ofevent based on data identifying the unauthorized subject, the matchingscore referring to a degree of correlation between the data identifyingthe unauthorized subject and data identifying the authorized subject.The method may further comprise a step of determining if an event of thelist of events relating to the unauthorized subject has a matching scorelower than a first matching threshold, i.e. unauthorized subjectmatching threshold, the matching score lower than the first matchingthreshold indicating that the event of the list of events relates to theunauthorized subject. According to the present disclosure, at least oneevent of the list of events relating to the unauthorized subject has amatching score lower than the first matching threshold. The method maycomprise a step of assigning the at least one event with an unauthorizedsubject identifier (unauthorized subject ID) and based on the at leastone event, forming the list of events relating to the unauthorizedsubject.

According to an embodiment, at step 202, the method may further comprisea step of determining if an event of the list of events relating to theauthorized subject has a matching score higher than a second matchingthreshold, i.e. authorized subject matching threshold, the matchingscore higher than the second matching threshold indicating that theevent of the list of events relates to the authorized subject. Accordingto the present disclosure, at least one event has matching score higherthan the second matching threshold. The method may comprise a step ofassigning the at least one event with an authorized subject identifier(authorized subject ID) and based on the at least one event, forming thelist of events relating to the authorized subject.

According to an embodiment, at step 202, the method may further comprisea step of calculating a matching score of the event based on dataidentifying the unauthorized subject, and a step of determining if thematching score of the event is higher than the first matching thresholdand lower than the second matching threshold. In various embodiments,events with matching score higher than the first matching threshold andlower than the second matching threshold may refer to a non-identifiedevent, and may be retrieved and will be assigned with an authorizedsubject ID or unauthorized subject ID through steps 202 and 204. Inparticular at step 204, the method may comprise a step of determining anon-identified event relates to the unauthorized subject in response todetermining the likelihood of how the non-identified event is similar tothe at least one event of the list of event relating to the unauthorizedsubject higher than the likelihood of how the non-identified event issimilar to the at least one event of the list of event relating to theunauthorized subject. As such, the method may then further comprise astep of assigning the non-identified event with an unauthorized subjectID and adding this newly assigned event in the list of events with theunauthorized subject ID relating to the unauthorized subject.

According to an embodiment, at step 204, the method may comprise a stepof determining the non-identified event to relate to the authorizedsubject in response to determining the likelihood of how the event issimilar to the at least one event of the list of event relating to theauthorized subject higher than the likelihood of how the event issimilar to the at least one event of the list of event relating to theunauthorized subject. As such, the method may then further comprise astep of assigning the non-identified event with an authorized subject IDand adding this newly assigned event in the list of events with theauthorized subject ID relating to the authorized subject.

According to an embodiment, at step 204, the method may further comprisea step of updating the data identifying the authorized subject tocomprise the data identifying the unauthorized subject of each event ofthe list of events relating to the authorized subject. The updated dataidentifying the authorized subject may be used for calculating matchingscore of a subsequent event identified based on the input of asubsequent time period and determining if the subsequent event relatesto an unauthorized subject.

FIG. 3 depict a block diagram illustrating a system 300 for determiningif an event relates to an unauthorized subject according to anembodiment. In an example, the managing of input is performed by atleast an image capturing device 302. The system 300 comprises an imagecapturing device 302 in communication with the apparatus 304. In animplementation, the apparatus 304 may be generally described as aphysical device comprising at least one processor 306 and at least onememory 308 including computer program code. The at least one memory 308and the computer program code are configured to, with the at least oneprocessor 306, cause the physical device to perform the operationsdescribed in FIG. 2 . The processor 306 is configured to receive animage from the image capturing device 302 or to retrieve image from adatabase 310.

The image capturing device may be a device such as closed-circuittelevision (CCTV), web-cams, surveillance camera, or other similardevices, which provide a variety of information of which characteristicinformation and time information that can be used by the system toidentify a subject and obtain data identifying the subject. In animplementation, the characteristic information derived from the imagecapturing device 302 for identifying a subject and obtaining dataidentifying the subject may include physical characteristic informationsuch as height, body size, hair colour, skin colour, facial feature,apparel, belongings, other similar characteristic or combinations, orbehavioral characteristic information such as body movement, position oflimbs, direction of movement, the way of a subject walks, stands, movesand talks, other similar characteristic or combination. For example,facial information may be used to identify a subject while othercharacteristic information associated with the subject may be obtainedand aggregated as data identifying the subject, and stored in memory 308of the apparatus 304 or a database 310 accessible by the apparatus 304.In an implementation, the time information derived from the imagecapturing device and/or sensor 302 may be include timestamp at whicheach image or data is identified. The timestamps of images and data maybe stored in memory 308 of the apparatus 304 or a database 310accessible by the apparatus 304 to identify an event referring to anappearance of a subject and aggregate with the characteristicinformation of the subject as data identifying the subject such that theevent can be searched and retrieved based on time information. It shouldbe appreciated that the database 310 may be a part of the apparatus 304.

The apparatus 304 may be configured to communicate with the imagingcapturing device 302 and the database 310. In an example, the apparatus304 may receive, from the image capturing device, or retrieve from thedatabase 310, an image as input, and after processing by the processor306 in apparatus 304, identify an appearance of an unauthorized subjectand data identifying the unauthorized subject, and generate an outputsuch as an event comprising the data identifying the unauthorizedsubject which may be used for determining if the event relates to theunauthorized subject. During the specific time period, more than oneevents may be detected if more than one appearances of the unauthorizedsubject are identified based on the input of one or more images capturedfrom the image capturing device 302 or the database 310, and each of themore than one event comprising respective data identifying theunauthorized subject obtained based on the input and is determined ifthe event relates to the unauthorized subject.

In various embodiments below, after receiving an image from the imagecapturing device 302, or retrieving an image from the database 310, thememory 308 and the computer program code stored therein are configuredto, with the processor 306, cause the apparatus 304 to determine alikelihood of how the event is similar to at least one of: (i) at leastone event of a list of events relating to the unauthorized subject and(ii) at least one of a list of events relating to an authorized subject,each event of the list of events comprising data identifying theunauthorized subject, the determination of likelihood being based on thedata identifying the unauthorized subject of the event based on theimage; and determine the event to relate to the unauthorized subject inresponse to the determination of the likelihood.

The apparatus 304 may be further configured to calculate a matchingscore of each event of the list of events based on data identifying theunauthorized subject obtained from the image input; and determine if anevent of the list of events relating to the unauthorized subject has amatching score lower than a first matching threshold, i.e. unauthorizedsubject matching threshold, the matching score lower than the firstmatching threshold indicating that the event of the list of eventsrelates to the unauthorized subject. According to the presentdisclosure, the apparatus 304 may be configured to determine at leastone event of the list of events relating to the unauthorized subjecthaving a matching score lower than the first matching threshold, andassign the at least one event of the list of events with an unauthorizedsubject ID. Each event of the list of events with the unauthorizedsubject ID relating to the unauthorized subject is stored in thedatabase 310 and is retrieved for unauthorized subject detection andsubject identifier propagation process.

The apparatus 304 may be further configured to determine if an event ofthe list of events relating to the authorized subject has a matchingscore higher than a second matching threshold, i.e. authorized subjectmatching threshold, wherein the second matching threshold is higher thanthe first matching threshold, the matching score higher than the secondmatching threshold indicating the event of the list of events relates tothe authorized subject. According to the present disclosure, theapparatus 304 may be configured to determine at least one event of thelist of events relating to the authorized subject having a matchingscore higher than the second matching threshold, and assign the at leastone event of the list of events with an authorized subject IDcorresponding to the authorized subject. Each event of the list ofevents with the authorized subject ID relating to the authorized subjectis stored in the database 310 and is retrieved for unauthorized subjectdetection and subject identifier propagation process.

The apparatus 304 may be further configured to calculate a matchingscore of the event based on data identifying the unauthorized subject;and determining if the matching score of the event is higher than thefirst matching threshold, and lower than the second matching threshold.In various embodiments, such event may be retrieved from the database310 as an non-identified event and processed by the processor 306 inapparatus 304 to determine if the non-identified event relates to anunauthorized subject. In particular, the apparatus 304 may be configuredto, in response to determine the likelihood of how the non-identifiedevent is similar to the at least one event of the list of event relatingto the unauthorized subject higher than the likelihood of how thenon-identified event is similar to the at least one event of the list ofevent relating to the authorized subject, determine the non-identifiedevent to relate to the unauthorized subject, assign the non-identifiedevent with the unauthorized subject ID and add this newly assigned eventin the list of events relating to the unauthorized subject.

According to another embodiment, the apparatus 304 may be configured to,in response to determine the likelihood of how the non-identified eventis similar to the at least one event of the list of event relating tothe authorized subject higher than the likelihood of how thenon-identified event is similar to the at least one event of the list ofevent relating to the unauthorized subject, determine the non-identifiedevent to relate to the authorized subject, assign the non-identifiedevent with the authorized subject ID and add this newly assigned eventin the list of events relating to the authorized subject.

The apparatus 304 may be further configured to update the dataidentifying the authorized subject in the database 310 with the dataidentifying the unauthorized subject of each event of the list of eventsrelating to the authorized subject. The apparatus 304 may be configuredto retrieve such updated data identifying the authorized subject andused the updated data for calculating of matching score of a subsequentevent identified based on an input of a subsequent time period anddetermining if the subsequent event relates to an unauthorized subject.

FIG. 4 depicts the system 300 with more details. In this embodiment, theapparatus 304 may comprise a subject detection module 426, an onlinesame subject clustering module 428, appeared subject repository 430,appeared subject retrieval module 432, subject identifier assigningmodule 434, subject identifier repository 436, subject identifierpropagation module 438 and unauthorized subject retrieval module 440.The subject detection module of the apparatus 304 is configured toreceive images for identifying an appearance of an unauthorized subjectand an event comprising data identifying the unauthorized subject basedon an image input 443 during a specific time period. The online samesubject clustering module 428 is configured to receive events from thesubject detection module 426 and search for events with similarappearances or data identifying the unauthorized subject detected basedon the input and assigned a subject identifier (subject ID) to eventswith similar appearances. Each appearance will be assigned to a subjectID and stored in the appeared subject repository 430. More informationregarding the process of online same subject clustering will bediscussed in FIG. 10 . An example implementation of the appeared subjectrepository 430 is shown in table 1. It should be appreciated that theappeared subject repository 430 may be implemented as part of the memory308 or a database 310 within the apparatus 304 as shown in FIG. 3 . Theappeared subject retrieval module 432 is configured to retrieve eventsof a specific time period assigned with the subject ID from the appearedsubject repository 430 for further analysis such as subject identifierpropagation. In an embodiment, a search request 444 is used to signalthe appeared subject retrieval module 432 to retrieve the events of thespecific time period for further analysis. In another embodiment, thesignaling to the appearance generated by the search request can bescheduled such that the retrieval of events is carried out at a specifictime or after every specific time period for further analysis.

The subject identifier assigning module 434 is configured to receiveevents from the subject detection module 426, and determine if an eventrelates to an authorized subject based on appearance or data identifyingthe unauthorized subject obtained from the image input 422. In responseto determining the event to relate to an authorized subject, the subjectidentifier assigning module 434 is configured to assign the event withan authorized subject ID corresponding to the authorized subject. In anembodiment, the subject identifier assigning module 434 may beconfigured to calculate a matching score by matching data identifyingthe unauthorized subject obtained from the image input 422 against dataidentifying the authorized subject retrieved from database 310,determine the event to relate to the authorized subject based on thematching score exceeding the second matching threshold, i.e. authorizedsubject matching threshold and assign an authorized subject IDcorresponding to the authorized subject. In another embodiment, wherethere is a list of authorized subjects in the database 310, the subjectidentifier assigning module 434 may be configured to calculate acorresponding plurality of matching scores by comparing the dataidentifying the unauthorized subject against data identifying each ofthe list of authorized subjects in the database 310, and if more thanone matching scores exceed the second matching threshold, the subjectidentifier assigning module 434 may be configured to select theauthorized subject who provides the highest matching score among theplurality of matching scores, assign an authorized subject IDcorresponding to the authorized subject such that the event relates tothe authorized subject of the highest matching score. An exampleimplementation of subject identifier repository 436 is shown in table 2.It should be appreciated that the subject identifier repository 436 maybe implemented as part of the memory 308 or a database 310 within theapparatus 304 as shown in FIG. 3 .

According to the present disclosure, the subject identifier assigningmodule 434 may also be configured to determine if an event relates to anunauthorized subject based on appearance or data identifying theunauthorized subject obtained from the image input 422. In response todetermining the event to relate to the unauthorized subject, the subjectidentifier assigning module 434 is configured to assign the event withan unauthorized subject identifier corresponding to the unauthorizedsubject. Similarly, the subject identifier assigning module may beconfigured to determine the event to relate to the unauthorized subjectbased on a matching score of the event, in particular, the matchingscore falling below the first matching threshold, i.e. unauthorizedsubject matching threshold, wherein the second matching threshold ishigher than the first matching threshold In an embodiment, when two ormore events have a matching score falling below the first matchingthreshold, the two or more events will be assigned with a sameunauthorized subject identifier, and they will be aggregated to a listof event relating to the unauthorized subject.

The subject identifier propagation module 438 is configured to performsubject identifier propagation process based on the input of events withauthorized subject ID and the input of events with subject IDs detectedduring a specific time period from subject identifier repository 436 andappeared subject retrieval module 432 respectively. According to thepresent disclosure, the subject identifier propagation module 438 may beconfigured to assign each non-identified event of the subject IDs withthe authorized subject ID or the unauthorized subject ID by determininga likelihood on how the non-identified event is similar to theidentified events. The unauthorized subject retrieval module 440 isconfigured to retrieve events relating to the unauthorized subject withthe unauthorized subject ID and non-identified events, if any, togenerate alerts 442 indicating the detection of the unauthorizedsubject. Examples of implementation of subject identifier propagationmodule 438 and unauthorized subject retrieval module 440 are plotted intables 3 and 4 respectively.

In an example implementation of the system 300, according to tables 1 to4, an event is detected by subject detection module 426 based on subjectappearances through facial information obtained from image input 422 oftwo image capturing devices, i.e. cameras 1 and 2.

The eight events corresponding to eight subject appearances areindicated as data ID (face ID) 1 to 8 respectively. Based on thesimilarity in the appearances, the eight events can be assigned to twosubject identifiers (subject IDs 1 and 2) corresponding to two differentdetected subjects by online same subject clustering module 428 andstored in appeared subject repository 430, as shown in table 1 togetherwith respective locations characterized by the camera from which theimage was captured and time indicating the timestamp in which the imagewas captured. A table 1 is an example implementation of an appearedperson repository.

TABLE 1 Data ID (face ID) Subject ID Location Time 1 1 Camera 1 10:00:00AM, 28 Dec. 2019 2 2 Camera 1 11:00:40 AM, 28 Dec. 2019 3 1 Camera 111:05:20 AM, 28 Dec. 2019 4 1 Camera 1 11:05:21 AM, 28 Dec. 2019 5 2Camera 2 12:15:10 AM, 28 Dec. 2019 6 2 Camera 2 12:15:16 AM, 28 Dec.2019 7 2 Camera 2 12:15:23 AM, 28 Dec. 2019 8 1 Camera 2 12:16:00 AM, 28Dec. 2019

On the other hand, each event is processed by the subject identifierassigning module 434 to determine if the event relate to an authorizedsubject based on its data identifying the subject. In this example,events of Data IDs 2 and 6 under subject ID 2 are determined to relateto an authorized subject Yamazaki and therefore assigned with anauthorized subject ID corresponding to the authorized subject. Theevents are stored in subject identifier repository 436 with theinformation of authorized subject ID, as shown in table 2. In anembodiment, the determination of an event relating to an authorizedsubject is based on a matching score of the event by matching dataidentifying the subject obtained from the image input 422 against dataidentifying the authorized subjects, e.g. Yamazaki, in the database 310,the matching score exceeding the authorized subject matching threshold.In this example, the other 2 events under subject ID 2, i.e. Data IDs 5and 7, has a matching score not exceeding the authorized subjectmatching threshold, hence events of Data IDs 5 and 7 are not assignedwith the authorized subject ID of Yamazaki and the events remainnon-identified in the processing of the subject identifier assigningmodule 434. Similarly, none of the events under subject ID 1 comprisesdata that match data identifying any of the authorized subject or amatching score exceeding the authorized subject matching threshold. As aresult, the events under subject ID 1 are not assigned with anyauthorized subject ID and remain non-identified in the processing of thesubject identifier assigning module 434. A table 2 is an exampleimplementation of a subject identifier repository.

TABLE 2 Data ID Authorized (face ID) subject ID 1 — 2 Yamazaki 3 — 4 — 5— 6 Yamazaki 7 — 8 —

Additionally or alternatively, each event may be determined to relate tothe unauthorized subject based on a matching score falling below theunauthorized subject matching threshold. In this example, event of DataID 7 under subject ID 2 has a matching score falling below theunauthorized subject matching threshold, hence event of Data ID 7 areassigned with an unauthorized subject ID. The event may be stored in thesubject identifier repository 436 with the unauthorized subject IDinformation (not shown in table 2).

Upon triggered by a search request 444, the appeared subject retrievalmodule 432 may retrieve events under a subject ID, e.g. events of DataIDs 2, 5, 6 and 7 under subject ID 1, and send to subject identifierpropagation module 438. Subject identifier propagation module 438 thenperform subject identifier propagation process based on the eventsretrieved from the appeared subject retrieval module 432 andauthorized/unauthorized subject ID information obtained from the subjectidentifier repository 436 by determining a likelihood of how annon-identified event, e.g. events of Data ID 5, is similar to the eventwith the authorized subject ID, e.g. events of Data IDs 2 and 6 relatingto the authorized subject Yamazaki, and/or the event with theunauthorized subject ID, e.g. event Data ID 7. In this example, it isdetermined that the likelihood of how the non-identified event issimilar to the event with the authorized subject ID is higher than thelikelihood of how the non-identified event is similar to the event withthe unauthorized subject ID, and as a result, event of Data ID 5 isdetermined to relate to the authorized subject and assigned with theauthorized subject ID, as illustrated in table 3. In an embodiment, thedetermination of likelihood may be based on its matching score, inparticular, how close is the matching score between the non-identifiedevent to the identified events. In this example, event of Data ID 5 mayhave a matching score closer to that of event of Data ID 2 or 6 thanthat of event of Data ID 7, therefore the authorized subject ID of eventof Data ID 2 or 6 is assigned to the event of Data ID 5. According tovarious embodiments, the events with authorized subject ID will beaggregated to form a list of event relating to the authorized subject.Likewise, events with unauthorized subject ID will be aggregated to forma list of events relating to the unauthorized subject. In this example,after the event of Data ID 5 is determined to relate to the authorizedsubject and assigned with the authorized subject ID, the event of DataID 5 will be added to a list of events relating to the authorizedsubject comprising the events of Data IDs 2, 5 and 6. A table 3 is anexample implementation of a subject identifier propagation module.

TABLE 3 Data ID Author i zed (face ID) subject ID Identifier type 2Yamazaki Authorized subject ID 5 — Authorized subject ID 6 Yamazak iAuthorized subject ID 7 — Unauthorized subject ID

According to the present disclosure, in order for subject identifierpropagation module 438 to operate and initiate subject identifiedpropagation process, the system requires the retrieved events under thesubject ID to have at least one event that has been assigned with theauthorized subject ID and at least one event that has been assigned withthe unauthorized subject ID. In this regard, as none of the events undersubject ID 1 is assigned with an authorized subject ID in the processingof the subject identifier assigning module 434, subject identifierpropagation module 438 may not be able to perform subject identifierpropagation process on the events under subject ID 1. As a result,events of Data IDs 1, 3, 4 and 8 under subject ID 1 may remainnon-identified in the processing of subject identifier propagationmodule 438.

After the subject identifier propagation process, the non-identifierevents and events with unauthorized subject ID, namely events of DataIDs 1, 3, 4, 7 and 8, are retrieved by unauthorized subject retrievalmodule 440, as shown in table 4. Each of the non-identified events orevents with unauthorized subject ID may generate an alert 442 toindicate a detection of an unauthorized subject. A table 4 is an exampleimplementation of a non-identified subject retrieval module.

TABLE 4 Data ID (face ID) Subject ID Location Time 1 1 Camera 1 10:00:00AM, 28 Dec. 2019 3 1 Camera 1 11:05:20 AM, 28 Dec. 2019 4 1 Camera 111:05:21 AM, 28 Dec. 2019 7 2 Camera 2 12:15:23 AM, 28 Dec. 2019 8 1Camera 2 12:16:00 AM, 28 Dec. 2019

FIG. 5 depict a flow chart 500 illustrating a process of unauthorizedsubject retrieval according to an embodiment. Prior to the retrievingevents with unauthorized subject ID, at step 502, all eventscorresponding to all subject appearances and subject IDs during aspecific time period are retrieved. At step 504, it is determined if allthe events have been checked. In an embodiment, an event is determinedas checked if the event has been assigned a subject ID, or has beendetermined if a matching score of the event is higher or lower than thefirst matching threshold and/or the second matching threshold forexample by online same subject clustering module 428, and stored in theappeared subject repository 430. If all the events are not been checked,step 506 is carried out. At step 506, an event which has not beenchecked is picked up and processed by the online same subject clusteringmodule 428. At step 508, it is determined if the event relates to anauthorized subject, i.e. its matching score is higher than the secondmatching threshold. If it is relates to an authorized subject, step 510is carried out. At step 510, the event is assigned with an authorizedsubject ID corresponding to the authorized subject. The process thenfurther comprises applying subject identifier propagation to all eventsbelonging to the subject, i.e. under the same subject ID. In anembodiment, prior to applying subject identifier propagation to all theevents belonging to the subject, the process may comprise determining atleast one of all events belonging to this subject to relate to anunauthorized subject, i.e. determining its matching score being lowerthan the first matching threshold and assigning it with an unauthorizedsubject ID. In the subject identifier propagation process, all eventsbelonging to the subject will be assigned with an authorized subject IDor an unauthorized subject ID. More information on the process andmethod of subject identifier propagation will be discussed in FIGS. 6 to6E in the following.

Returning to step 508, if the event does not relate to an authorizedsubject, the process is directed to step 504. In an embodiment, as aresult of subject identifier propagation process, events with theauthorized subject ID are aggregated to a list of events relating to theauthorized subject, whereas event with the unauthorized subject ID areaggregated to a list of events relating to the unauthorized subject.Returning to step 504, if all the events with all subject IDs have beenchecked, the process of non-identified subject retrieval is carried outat step 512 to retrieve all events with unauthorized subject ID.Subsequently, for every list of event assigned under an unauthorizedsubject ID relating to an unauthorized subject, an alert is generated toindicate the detection of the unauthorized subject at step 514, and theprocess may end.

FIG. 6 depicts a flow chart 600 illustrating a process of subjectidentifier propagation according to an embodiment. Prior to performingthe subject identifier propagation, at step 602, all events comprisingdata identifying a subject, e.g. subject ID 1, are retrieved. At step604, if one or more event has a matching score lower than the firstmatching threshold, i.e. unauthorized subject matching threshold, anunauthorized subject ID is assigned to the one or more event. At step606, a subject identifier propagation algorithm is performed to assignthe unauthorized subject ID or the authorized subject ID, to allnon-identified events. In case where an non-identified event isdetermined as noise, the event will not be assigned to the unauthorizedsubject ID or the authorized subject ID and will remain non-identified.

FIG. 7 depicts diagrams illustrating the flow chart 600 of the processof subject identifier propagation. Specifically, FIG. 7(a) shows a graphcontaining a total of 12 events comprising data identifying a subject,e.g. subject ID 1, identified during a specific time period based onappearances of the subject. The axes of the graph may be related tomatching score, where events with similar data identifying the subjector similar matching score may be plotted closer to each other in thegraph, whereas events with dissimilar data identifying the subject ordifferent matching score may be plotted further apart from each other inthe graph. In an embodiment, an event among the 12 events may bedetermined to relate to an authorized subject, for example, it has amatching score higher than the second matching threshold as a result ofmatching its data identifying the subject against data identifying theauthorized subject. Such event like event 612 is assigned with anauthorized subject ID as shown in FIG. 7(a). In an embodiment, the event612 with the authorized subject ID may be used as a centre point of thegraph, where events comprising similar data identifying the subject tothat of event 612 is plotted closer to the event 612 whereas eventscomprising dissimilar data identifying the subject will be plottedfurther to the event 612 and closer to the edge of the graph.

FIG. 7(b) illustrates a step of assigning unauthorized subject ID to anevent which has a matching score lower than the first matchingthreshold. In this embodiment, a circle 614 may be used to represent thefirst matching threshold, wherein an event falling outside the circle614 is characterized as having a matching score lower than the firstmatching threshold (circle 614), for example events 616 and 618. As aresult, events 616 and 618 will be assigned with the unauthorizedsubject ID.

FIG. 7(c) illustrates a subject identifier propagation process. Duringthe subject identifier propagation process, for each identified eventsuch as events 612, 616 and 618, a neighbouring event search may beconducted to search for neighbouring events comprising similar dataidentifying the subject. If a neighbouring event is a non-identifiedevent, the non-identified event may be assigned with the ID of theidentified event; as a result, the ID is propagated to the neighbouringevent. In particular, due to the similarity in data identifying thesubject, an event that is adjacent or close to an identified event has agreater likelihood to be determined to relate to the identified eventand assigned with the ID of the identified event in the subjectidentifier propagation process. The subject identifier propagationprocess may repeat until all the events under the subject ID areassigned with either an authorized subject ID or an unauthorized subjectID. In case where an non-identified event is determined as noise forexample the event could not be reached by a neighboring event search ofany identified event (not shown in the present disclosure), the eventwill remain non-identified.

In this embodiment, the subject identifier propagation process isinitiated with identified events 612, 616 and 618. In the first round ofsubject identifier propagation process, events adjacent to the events612 such as event 621 may be reached by the neighbouring event search ofevent 612 and assigned with an authorized subject ID, similarly, events622 and 623 may be reached by the neighbouring event search of events616 and 618 respectively and assigned with the unauthorized subject ID.The events with the same assigned ID is then aggregated to form a listof event for example, a list of event relating to the authorized subjectcomprising events 612 and 621, a list of events relating to theunauthorized subject comprising events 616, 618, 622 and 623.Subsequently, the subject identifier propagation may continue usingidentified events 612, 616, 618, 621, 622, 623.

In this second round of subject identifier propagation process, eventsadjacent to the identified events 612, 621, 622 such as events 624, 625and 626 may be reached by the search and assigned to respective IDs andaggregated into the list of events comprising the identified event. Inparticular, events 624 and 625 are assigned based on events 621 and 612respectively the authorized subject ID and added into the list of eventsrelating to the authorized subject; whereas event 626 is assigned basedon event 621 the unauthorized subject ID and added into the list ofevents relating to the unauthorized subject. In this embodiment, thereis no non-identified event in the neighbouring event search of events618 and 623, so further subject identifier propagation process may notbe performed for events 618 and 623. Similarly, in the third round ofsubject identifier propagation, events 627, 628 may be determined torelate to the authorized subject based on identified events 625 andassigned with the authorized subject ID; whereas events 629 may bedetermined to relate to the unauthorized subject based on identifiedevent 626 and assigned with the unauthorized subject ID. In anembodiment, such subject identifier propagation process can be achievedby executing equation 2 iteratively, which will be described further inthe following.

The subject identifier propagation process may end when all events underthe subject ID are assigned with the authorized or unauthorized subjectID. At the end of the subject identifier propagation process, it may beresulted with a list of event relating to the authorized subjectcomprises events 612, 621, 622, 625, 628 and 629 and a list of eventrelating to the unauthorized subject comprises events 616, 618, 622,623, 626 and 629. In various embodiments, the list of events relating tothe unauthorized subject will then be used to generate an alertindicating the detection of the unauthorized subject.

Advantageously, the present disclosure manages events which do not matchwith an authorized subject (all events except event 612) and processthem with subject identifier propagation process, such that alerts thatare not supposed to be generated are reduced.

Conventionally, once an event is determined that it does not relate toany authorized subject such as events 621, 624, 625, 627 and 628, theevent may be used to generate an alert. In contrast, according to thepresent disclosure, such events such as events 621, 624, 625, 627 and628 are managed by determining if they relate to the authorized subjectthrough subject identifier propagation process; as a result, the alertsare not generated.

In an embodiment, a density-based spatial clustering of applicationswith noise (DBSCAN) may be used as the algorithm to achieve the subjectidentifier propagation described in FIG. 7(c). Specifically, by settinga radius of neighbouring event search, i.e. (eps) and minimum number ofevents within the search area, i.e. minPts, the algorithm is configuredto construct indexed database for radius of the neighbouring eventsearch, find the neighbouring events and identify an identified event(with authorized subject ID or unauthorized subject ID) within thesearch area, find the connected components of identified events,ignoring all non-identified events, and assign each non-identified eventto a nearby cluster, e.g. a list of identified events, if the clustercomprise a neighbouring event, otherwise assign the non-identifiedevents to noise. FIG. 8 depicts diagram illustrating an example subjectidentifier propagation process 630. Initially at step 631, a pluralityof events may be identified based on an image input during a specifictime period with a same subject identifier (subject ID), for example, byonline same subject clustering module. Each of the plurality of eventsis used to match against data identifying each subject of a list ofauthorized subjects. In this embodiment, it is determined that event 638relates to an authorized subject 636 of the list of authorized subjectand is thus assigned with an authorized subject ID. A circle 640 may beused to represent the second matching threshold, i.e. authorized subjectmatching threshold, wherein each event which falls inside the circle 640is determined as having a matching score higher than the authorizedsubject threshold (circle 640) and may be similarly assigned with theauthorized subject ID. All events within the circles are aggregated toform a list of events relating to the authorized subject 636. On theother hand, based on a matching score falling below the first matchingthreshold, events 642 and 644 may be determined to relate to anunauthorized subject and are assigned with an unauthorized subject ID.

The subject propagation is illustrated at step 632, where eventsadjacent to the identified events have a greater likelihood to bedetermined to relate to the identified events and assigned with the IDof the identified events in the subject identifier propagation processdue to the similarity in data identifying the subject. In particular,events 648 are determined to relate the authorized subject 636, thusthey are assigned to the authorized subject ID and are added to the listof events relating to the authorized subject 636. Similarly, the subjectidentifier propagation process may determine events 652 and 654 torelate to the unauthorized subject due to proximity of data identifyingthe unauthorized subject, assign the events 652 and 654 with theunauthorized subject ID and add the events 652 and 654 to the list ofevents relating to the unauthorized subject. At step 633, the list ofevents relating to the unauthorized subject may be used generate analert indicating the detection of the unauthorized subject.

Various embodiments provide system with portion of the database or thesubject identifier repository are used to stored data identifying eachsubject in the list of authorized or known unauthorized subjects fordetermining if an event relates to an unauthorized subject. FIG. 9depicts an example system 700 for determining if an event relates to anunauthorized subject based on an input of an image capturing device 702.The database or the subject identifier repository 436 may be used tostore data identifying each authorized subject, e.g. known subject ID 1to 4 after an event of a subject appearance identified based on theimage capturing device 702 is determined to relate to the authorizedsubject. In particular, appearance of subject 704, 706 may be identifiedby the image capturing device 702, and an event comprising dataidentifying the subject 704 a, 706 a may be determined to relate to anauthorized subject, namely known subject IDs 1 and 2 in the database orsubject identifier repository 436 respectively. The data identifying thesubjects 704 a and 706 a is then stored in the portion of the databaseor the subject identifier repository directed to known subject IDs 1 and2, e.g. 436 a and 436 b respectively, to update the data identifying theauthorized subject for subsequent subject detection and subjectidentifier propagation process.

In an embodiment, the system 700 may be implemented to include a knownunauthorized subject (for example known subject ID 100000) in the listof known subjects stored in the database or subject identifierrepository 436 for subject detection and subject identifier propagationprocess. A known unauthorized subject may be a subject that isidentified by an investigator 720 and stored in the memory for crimeprevention and investigation. In this embodiment, appearance of subject708 may be identified by the image capturing device 702, and an eventcomprising data identifying the subject 708 a may be determined torelate to the known authorized subject of known subject ID 100000 in thedatabase or subject identifier repository 436. The event comprising dataidentifying the subject 708 a may also be retrieved by the system andused in subject identifier propagation process, and a list of event 708b with a known unauthorized subject identifier relating to the knownunauthorized subject may be determined and used to generate an alert tonotify the investigator 720 of the detection of the known unauthorizedsubject. Similarly, the data identifying the subject 708 a is stored inthe portion of the database or the subject identifier repositorydirected to known subject ID 100000, e.g. 436 c, to update the dataidentifying the known unauthorized subject for subsequent subjectdetection and subject identifier propagation process.

FIG. 10 depicts a flow chart illustrating a process of online samesubject clustering according an embodiment. The process may start atstep 802, where appearances of subjects are detected based on an input.An event may be identified for each detected subject appearance. At step804, the process may undergo a search of similar event, for examplebased on similar detected subject appearance or data identifying asubject, which has similarity score higher than a similarity score lowerbound. At step 806, it is then determined if the number of similarevents is higher than a lower bound of number of similar events. If thenumber of similar events is not higher than the lower bound of number ofsimilar events, step 816 is carried out. At step 816, a new subjectinformation, e.g. subject identifier (subject ID), related to thesimilar events is created on appeared subject repository. Returning tostep 806, if it is determined that the number of similar events ishigher than the lower bound of number of similar events, step 808 iscarried out. At step 808, it is determined if all similar events areassigned to a same subject ID. If all similar events are assigned to asame subject ID, step 810 is carried out to input the events with thesame subject ID to the appeared subject repository. If all similarevents are not assigned to a same subject ID, the online same personclustering process will merge existing subject information such that thesimilar events are assigned to a same subject ID. For example, if one ofthe similar events has a subject ID of 3 and the rest of the similarevents have a subject ID of 1, the similar events may be assigned tosubject ID 1 or to a new subject ID 4. This will ensure all similarevents will be retrieved collectively for subsequent subject identifierpropagation process as shown in FIG. 6 . Subsequently, the subjectinformation is added to appeared subject repository at step 810. At step812, data related to the similar events such as location and time isinput and stored in the appeared subject repository, as shown in table1.

In another embodiment, a subject identifier propagation algorithm may beformulated as a constrained optimization problem into the equationbelow:

F({z _(i)})=Σ_(i)Σ_(j) R _(ij)(z _(i) −z _(j))²+λΣ_(i) ^(L)(y _(i) −z_(i))²  Equation (1)

where y_(i) is identifier score (1 or −1) given as training data, z_(i)(0<i<N, N is total number of events retrieved at step 602 on FIG. 6 ) issoft identifier score determined by this algorithm, R_(ij) is similarityscore between event data i and event data j, λ is the Lagrangemultiplier term, L is total number of identified events determined atstep 604 in FIG. 6 . “y_(i)=1” stands for “event data i is an authorizedsubject” and “y_(i)=−1” stands for “event data i is an unauthorizedsubject”. The constrained optimization problem defines the objectivefunction expected to be minimized which can be solved by taking thepartial derivatives of equation 1 in respect to λ and z_(i), i.e.

$\frac{\partial F}{\partial\lambda} = 0$ and$\frac{\partial F}{\partial z_{i}} = 0$

to form equation 2, and running iteratively until z_(i) converges.

$\begin{matrix}{z_{i} = {\frac{\sum_{j}{R_{ij}z_{j}}}{\sum_{j}R_{ij}}\left( {i > L} \right)}} & {{Equation}(2)}\end{matrix}$ $\begin{matrix}{z_{i} = {y_{i}\left( {i \leq L} \right)}} & {{Equation}(3)}\end{matrix}$

“z_(i)=y_(i)” is the constraint for identified events.

Each iteration takes a O(N²) computation complexity, where N is theinput size in units of bits needed to represent the input. By reducingN, the amount of time to run each iteration can be greatly reduced.According to the present disclosure, online same subject clustering mayadvantageously reduce N and thus the amount of time to run the subjectidentifier propagation algorithm.

FIG. 11 depicts a flow chart 900 illustrating a process of updatingsubject identifier repository according to an embodiment. In thisembodiment, facial information is used as data identifying a subject oran unauthorized subject from an image input and comparing against thelist of authorized subjects to determine if the event relates to anyauthorized subject in the list of authorized subjects. At step 902, anevent is identified based on a detected face of a subject 903 from aninput. At step 904, a search for similar subject over a list ofauthorized subjects 905 is conducted. The search for similar subject isconducted by comparing the facial information of the event against thefacial information of each subject of the list of authorized subjects905 and calculating a matching score indicating a degree of how thefacial information matches each other. A similar subject is detectedwhen the matching score is higher than the second matching threshold,i.e. authorized subject matching threshold. At step 906, it isdetermined if there is any similar subject in the list of authorizedsubjects 905. If there is a similar subject, the event comprising thefacial information identifying the subject 903 will be assigned with anauthorized subject ID corresponding to the similar (authorized) subject.Subsequently, the facial information comprised in the event with theauthorized subject ID may be stored in the subject identifier repositoryto update the facial information of the similar subject 905 a forsubsequent subject detection and subject identifier propagation process,as illustrated in FIG. 9 . In an embodiment, where more than one similarsubject in the list of authorized subject are identified, the processmay select one similar subject which provides the highest matching scoreamong all identified similar subjects, and the event comprising thefacial information identifying the subject will be assigned with anauthorized subject ID corresponding to the similar (authorized) subjectof the highest matching score. Returning to step 906, if there is nosimilar subject over the list of authorized subjects, the process mayend.

FIG. 12 depicts a flow chart 910 illustrating a process of updatingsubject identifier repository according to another embodiment. In thisembodiment, a subject 913 may be determined to relate to an authorizedsubject based on authorized clothes 915, and therefore body informationis used as data identifying a subject or an unauthorized subject from animage input to determine if the detected comprises authorized clothes.At step 912, an event of a subject appearance is identified based on adetected body of the subject from an input. At step 914, a matchingscore is calculated by comparing the body information of the eventagainst the authorized clothes. At step 916, it is determined if thematching score is higher than a matching threshold, for exampleauthorized subject matching threshold. If the matching score is higherthan the matching threshold, step 918 is carried out. At step 918, theevent is assigned an authorized clothes identifier. Subsequently, thebody information comprised in the event with the authorized clothesidentifier may be stored in the subject identifier repository to updatethe body information of the authorized clothes or authorized subject forsubsequent subject detection and subject identifier propagation process,as illustrated in FIG. 9 . It would appreciate any other bodyinformation such as apparel, belongings or even a behavioralcharacteristic of a subject such as body movement may be usedadditionally or alternatively in this embodiment as data identifying asubject or determining if an event relate to an authorized subject.

FIG. 13 depicts an exemplary computing device 1000, hereinafterinterchangeably referred to as a computer system 1000 or as a device1000, where one or more such computing devices 1000 may be used toimplement the system 300 shown in FIG. 4 . The following description ofthe computing device 1000 is provided by way of example only and is notintended to be limiting.

As shown in FIG. 13 , the example computing device 1000 includes aprocessor 1004 for executing software routines. Although a singleprocessor is shown for the sake of clarity, the computing device 1000may also include a multi-processor system. The processor 1004 isconnected to a communication infrastructure 1006 for communication withother components of the computing device 1000. The communicationinfrastructure 1006 may include, for example, a communications bus,cross-bar, or network.

The computing device 1000 further includes a primary memory 1008, suchas a random access memory (RAM), and a secondary memory 1010. Thesecondary memory 1010 may include, for example, a storage drive 1012,which may be a hard disk drive, a solid state drive or a hybrid driveand/or a removable storage drive 1014, which may include a magnetic tapedrive, an optical disk drive, a solid state storage drive (such as a USBflash drive, a flash memory device, a solid state drive or a memorycard), or the like. The removable storage drive 1014 reads from and/orwrites to a removable storage medium 1018 in a well-known manner. Theremovable storage medium 1018 may include magnetic tape, optical disk,non-volatile memory storage medium, or the like, which is read by andwritten to by removable storage drive 1014. As will be appreciated bypersons skilled in the relevant art(s), the removable storage medium1018 includes a computer readable storage medium having stored thereincomputer executable program code instructions and/or data.

In an alternative implementation, the secondary memory 1010 mayadditionally or alternatively include other similar means for allowingcomputer programs or other instructions to be loaded into the computingdevice 1000. Such means can include, for example, a removable storageunit 1022 and an interface 1020. Examples of a removable storage unit1022 and interface 1020 include a program cartridge and cartridgeinterface (such as that found in video game console devices), aremovable memory chip (such as an EPROM or PROM) and associated socket,a removable solid state storage drive (such as a USB flash drive, aflash memory device, a solid state drive or a memory card), and otherremovable storage units 1022 and interfaces 1020 which allow softwareand data to be transferred from the removable storage unit 1022 to thecomputer system 1000.

The computing device 1000 also includes at least one communicationinterface 1024. The communication interface 1024 allows software anddata to be transferred between computing device 1000 and externaldevices via a communication path 1026. In various embodiments of theinventions, the communication interface 1024 permits data to betransferred between the computing device 1000 and a data communicationnetwork, such as a public data or private data communication network.The communication interface 1024 may be used to exchange data betweendifferent computing devices 1000 which such computing devices 1000 formpart an interconnected computer network. Examples of a communicationinterface 1024 can include a modem, a network interface (such as anEthernet card), a communication port (such as a serial, parallel,printer, GPIB, IEEE 1394, RJ45, USB), an antenna with associatedcircuitry and the like. The communication interface 1024 may be wired ormay be wireless. Software and data transferred via the communicationinterface 1024 are in the form of signals which can be electronic,electromagnetic, optical or other signals capable of being received bycommunication interface 1024. These signals are provided to thecommunication interface via the communication interface 1024.

As shown in FIG. 13 , the computing device 1000 further includes adisplay interface 1002 which performs operations for rendering images toan associated display 1030 and an audio interface 1032 for performingoperations for playing audio content via associated speaker(s) 1034.

As used herein, the term “computer program product” (or computerreadable medium, which may be a non-transitory computer readable medium)may refer, in part, to removable storage medium 1018, removable storageunit 1022, a hard disk installed in storage drive 1012, or a carrierwave carrying software over communication path 1026 (wireless link orcable) to communication interface 1024. Computer readable storage media(or computer readable media) refers to any non-transitory, non-volatiletangible storage medium that provides recorded instructions and/or datato the computing device 1000 for execution and/or processing. Examplesof such storage media include magnetic tape, CD-ROM, DVD, Blu-ray(registered trademark) Disc, a hard disk drive, a ROM or integratedcircuit, a solid state storage drive (such as a USB flash drive, a flashmemory device, a solid state drive or a memory card), a hybrid drive, amagneto-optical disk, or a computer readable card such as a PCMCIA cardand the like, whether or not such devices are internal or external ofthe computing device 1000. Examples of transitory or non-tangiblecomputer readable transmission media that may also participate in theprovision of software, application programs, instructions and/or data tothe computing device 1000 include radio or infra-red transmissionchannels as well as a network connection to another computer ornetworked device, and the Internet or Intranets including e-mailtransmissions and information recorded on Websites and the like.

The computer programs (also called computer program code) are stored inprimary memory 508 and/or secondary memory 1010. Computer programs canalso be received via the communication interface 1024. Such computerprograms, when executed, enable the computing device 1000 to perform oneor more features of embodiments discussed herein. In variousembodiments, the computer programs, when executed, enable the processor1004 to perform features of the above-described embodiments.Accordingly, such computer programs represent controllers of thecomputer system 1000.

Software may be stored in a computer program product and loaded into thecomputing device 1000 using the removable storage drive 1014, thestorage drive 1012, or the interface 1020. The computer program productmay be a non-transitory computer readable medium. Alternatively, thecomputer program product may be downloaded to the computer system 1000over the communications path 1026. The software, when executed by theprocessor 1004, causes the computing device 1000 to perform functions ofembodiments described herein.

It is to be understood that the embodiment of FIG. 13 is presentedmerely by way of example. Therefore, in some embodiments one or morefeatures of the computing device 1000 may be omitted. Also, in someembodiments, one or more features of the computing device 1000 may becombined together. Additionally, in some embodiments, one or morefeatures of the computing device 1000 may be split into one or morecomponent parts. For example, the primary memory 1008 and/or thesecondary memory 1010 may serve(s) as the memory 308 for the apparatus304; while the processor 1004 may serve as the processor 306 of theapparatus 304.

It will be appreciated by a person skilled in the art that numerousvariations and/or modifications may be made to the present invention asshown in the specific embodiments without departing from the spirit orscope of the invention as broadly described. For example, the abovedescription mainly presenting alerts on a visual interface, but it willbe appreciated that another type of alert presentation, such as soundalert, can be used in alternate embodiments to implement the method.Some modifications, e.g. adding an access point, changing the log-inroutine, etc. may be considered and incorporated. The presentembodiments are, therefore, to be considered in all respects to beillustrative and not restrictive.

This software can be stored in various types of non-transitory computerreadable media and thereby supplied to computers. The non-transitorycomputer readable media includes various types of tangible storagemedia. Examples of the non-transitory computer readable media include amagnetic recording medium (such as a flexible disk, a magnetic tape, anda hard disk drive), a magneto-optic recording medium (such as amagneto-optic disk), a CD-ROM (Read Only Memory), a CD-R, and a CD-R/W,and a semiconductor memory (such as a mask ROM, a PROM (ProgrammableROM), an EPROM (Erasable PROM), a flash ROM, and a RAM (Random AccessMemory)). Further, the program can be supplied to computers by usingvarious types of transitory computer readable media. Examples of thetransitory computer readable media include an electrical signal, anoptical signal, and an electromagnetic wave. The transitory computerreadable media can be used to supply programs to computer through a wirecommunication path such as an electrical wire and an optical fiber, orwireless communication path.

The whole or part of the embodiments disclosed above can be describedas, but not limited to, the following supplementary notes.

(Supplementary Note 1)

A method for determining if an event relates to an unauthorized subject,the method comprising:

determining a likelihood of how the event is similar to at least one of:(i) at least one event of a list of events relating to the unauthorizedsubject and (ii) at least one of a list of events relating to anauthorized subject, each event of the list of events comprising dataidentifying the unauthorized subject, the determination of likelihoodbeing based on the data identifying the unauthorized subject; and

determining the event to relate to the unauthorized subject in responseto the determination of the likelihood.

(Supplementary Note 2)

The method of Supplementary note 1, further comprising:

calculating a matching score of each event of the list of events basedon data identifying the unauthorized subject, the matching scorereferring to a degree of correlation between the data identifying theunauthorized subject and data identifying the authorized subject; and

determining if an event of the list of events relating to theunauthorized subject has a matching score lower than a first matchingthreshold, the first matching threshold referring to a maximum matchingscore for an event to be determined as being related to the unauthorizedsubject, the matching score lower than the first matching thresholdindicating that the event of the list of events relates to theunauthorized subject.

(Supplementary Note 3)

The method of Supplementary note 2, further comprising:

determining if an event of the list of events relating to the authorizedsubject has a matching score higher than a second matching threshold,the second matching threshold referring to a minimum matching score foran event to be determined as being related to the authorized subject,the matching score higher than the second matching threshold indicatingthe event of the list of events relates to the authorized subject,wherein the second matching threshold is higher than the first matchingthreshold.

(Supplementary Note 4)

The method of Supplementary note 3, further comprising:

calculating a matching score of the event based on data identifying theunauthorized subject; and

determining if the matching score of the event is higher than the firstmatching threshold and lower than the second matching threshold, thematching score higher than the first matching threshold and lower thanthe second matching threshold indicating that the data identifying theunauthorized subject has a moderate degree of correlation with the dataidentifying the authorized subject.

(Supplementary Note 5)

The method of Supplementary note 1, wherein the step of determining theevent to relate to the unauthorized subject comprising:

determining the event to relate to the authorized subject in response todetermining the likelihood of how the event is similar to the at leastone event of the list of event relating to the authorized subject higherthan the likelihood of how the event is similar to the at least oneevent of the list of event relating to the unauthorized subject.

(Supplementary note 6)

The method in any one of Supplementary notes 2 to 5, further comprising:

updating the data identifying the authorized subject to comprise thedata identifying the unauthorized subject of each event of the list ofevents relating the authorized subject.

(Supplementary Note 7)

The method in any one of Supplementary notes 1 to 6, further comprising:

receiving an input, the input being at least one image captured by atleast one image capturing device during a specific time period, whereinthe detection of the event and the list of events comprising the dataidentifying the unauthorized subject is based on the received input.

(Supplementary Note 8)

An apparatus for determining if an event relates to an unauthorizedsubject, the apparatus comprising:

a memory in communication with a processor, the memory storing acomputer program recorded therein, the computer program being executableby the processor to cause the apparatus at least to:

determine a likelihood of how the event is similar to at least one eventof a list of events relating to the unauthorized subject, each event ofthe list of events comprising data identifying the unauthorized subject,the determination of likelihood being based on the data identifying theunauthorized subject; and

determine the event to relate to the unauthorized subject in response tothe determination of the likelihood.

(Supplementary Note 9)

The apparatus of Supplementary note 8, wherein the computer program isexecuted by the processor to cause the apparatus further to:

determine a likelihood of how the event is similar to at least one of:(i) at least one event of a list of events relating to the unauthorizedsubject and (ii) at least one of a list of events relating to anauthorized subject, each event of the list of events relating to theauthorized subject comprising data identifying the unauthorized subject,the determination of likelihood being based on the data identifying theunauthorized subject.

(Supplementary Note 10)

The apparatus of Supplementary note 9, wherein the computer program isexecuted by the processor to cause the apparatus further to:

determine if an event of the list of events relating to the authorizedsubject has a matching score higher than a second matching threshold,the second matching threshold referring to a minimum matching score foran event to be determined as being related to the authorized subject,the matching score higher than the second matching threshold indicatingthe event of the list of events relates to the authorized subject,wherein the second matching threshold is higher than a first matchingthreshold.

(Supplementary Note 11)

The apparatus of Supplementary note 10, wherein the computer program isexecuted by the processor to cause the apparatus further to:

calculate a matching score of the event based on data identifying theunauthorized subject; and

determine if the matching score of the event is higher than the firstmatching threshold and lower than the second matching threshold, thematching score higher than the first matching threshold and lower thanthe second matching threshold indicating that the data identifying theunauthorized subject has a moderate degree of correlation with the dataidentifying the authorized subject.

(Supplementary Note 12)

The apparatus of Supplementary note 8, wherein the computer program isexecuted by the processor to cause the apparatus further to:

determine the event to relate to the authorized subject in response todetermining the likelihood of how the event is similar to the at leastone event of the list of event relating to the authorized subject higherthan the likelihood of how the event is similar to the at least oneevent of the list of event relating to the unauthorized subject.

(Supplementary Note 13)

The apparatus in any one of Supplementary notes 9 to 12, wherein thecomputer program is executed by the processor to cause the apparatusfurther to:

update the data identifying the authorized subject to comprise the dataidentifying the unauthorized subject of each event of the list of eventsrelating the authorized subject.

(Supplementary Note 14)

The apparatus of any one of Supplementary notes 8 to 13, wherein thecomputer program is executed by the processor to cause the apparatusfurther to:

receive an input, the input being at least one image captured by atleast one image capturing device during a specific time period, whereinthe detection of the event and the list of events comprising the dataidentifying the unauthorized subject is based on the received input.

(Supplementary Note 15)

A system for determining if an event relates to an unauthorized subject,the system comprising:

the apparatus as Supplementary noted in any one of Supplementary notes 8to 14 and at least one image capturing device.

(Supplementary Note 16)

A non-transitory computer readable medium storing a program for causinga computer to execute:

determining a likelihood of how the event is similar to at least one of:(i) at least one event of a list of events relating to the unauthorizedsubject and (ii) at least one of a list of events relating to anauthorized subject, each event of the list of events comprising dataidentifying the unauthorized subject, the determination of likelihoodbeing based on the data identifying the unauthorized subject; and

determining the event to relate to the unauthorized subject in responseto the determination of the likelihood.

It will be appreciated by a person skilled in the art that numerousvariations and/or modifications may be made to the present disclosure asshown in the specific embodiments without departing from the spirit orscope of this disclosure as broadly described. The present embodimentsare, therefore, to be considered in all respects to be illustrative andnot restrictive.

This application is based upon and claims the benefit of priority fromSingapore patent application No. 10202000146S filed on 7 Jan. 2020, thedisclosure of which is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST

-   100 system-   101 a-101 e image-   102 list of authorized subjects-   102 a authorized subject-   103 alert-   104 no alert-   300 system-   302 image capturing device-   304 apparatus-   306 processor-   308 memory-   310 database-   426 subject detection module-   428 online same subject clustering module-   430 appeared subject repository-   432 appeared subject retrieval module-   434 subject identifier assigning module-   436 subject identifier repository-   438 subject identifier propagation module-   440 unauthorized subject retrieval module-   442 alerts-   443 image input-   444 search request-   612, 616, 618 event-   614 circle-   621-629 event-   636 authorized subject-   638 event-   640 circle-   642, 644, 648, 652, 654 event-   700 system-   702 image capturing device-   704, 704 a, 706, 706 a, 708, 708 a subject-   708 b event-   720 investigator-   436 a-436 c ID-   903, 905, 905 a subject-   913 subject-   915 authorized clothes-   1000 computing device-   1004 processor-   1006 communication infrastructure-   1008 primary memory-   1010 secondary memory-   1012 storage drive-   1014 removable storage drive-   1018 removable storage medium-   1020 interface-   1022 removable storage unit-   1024 communication interface-   1026 communication path-   1030 display-   1032 audio interface-   1034 speaker

What is claimed is:
 1. A method for determining if an event relates toan unauthorized subject, the method comprising: determining a likelihoodof how the event is similar to at least one of: (i) at least one eventof a list of events relating to the unauthorized subject and (ii) atleast one of a list of events relating to an authorized subject, eachevent of the list of events comprising data identifying the unauthorizedsubject, the determination of likelihood being based on the dataidentifying the unauthorized subject; and determining the event torelate to the unauthorized subject in response to the determination ofthe likelihood.
 2. The method of claim 1, further comprising:calculating a matching score of each event of the list of events basedon data identifying the unauthorized subject, the matching scorereferring to a degree of correlation between the data identifying theunauthorized subject and data identifying the authorized subject; anddetermining if an event of the list of events relating to theunauthorized subject has a matching score lower than a first matchingthreshold, the first matching threshold referring to a maximum matchingscore for an event to be determined as being related to the unauthorizedsubject, the matching score lower than the first matching thresholdindicating that the event of the list of events relates to theunauthorized subject.
 3. The method of claim 2, further comprising:determining if an event of the list of events relating to the authorizedsubject has a matching score higher than a second matching threshold,the second matching threshold referring to a minimum matching score foran event to be determined as being related to the authorized subject,the matching score higher than the second matching threshold indicatingthe event of the list of events relates to the authorized subject,wherein the second matching threshold is higher than the first matchingthreshold.
 4. The method of claim 3, further comprising: calculating amatching score of the event based on data identifying the unauthorizedsubject; and determining if the matching score of the event is higherthan the first matching threshold and lower than the second matchingthreshold, the matching score higher than the first matching thresholdand lower than the second matching threshold indicating that the dataidentifying the unauthorized subject has a moderate degree ofcorrelation with the data identifying the authorized subject.
 5. Themethod of claim 1, wherein the step of determining the event to relateto the unauthorized subject comprising: determining the event to relateto the authorized subject in response to determining the likelihood ofhow the event is similar to the at least one event of the list of eventrelating to the authorized subject higher than the likelihood of how theevent is similar to the at least one event of the list of event relatingto the unauthorized subject.
 6. The method in claim 2, furthercomprising: updating the data identifying the authorized subject tocomprise the data identifying the unauthorized subject of each event ofthe list of events relating the authorized subject.
 7. The method inclaim 1, further comprising: receiving an input, the input being atleast one image captured by at least one image capturing device during aspecific time period, wherein the detection of the event and the list ofevents comprising the data identifying the unauthorized subject is basedon the received input.
 8. An apparatus for determining if an eventrelates to an unauthorized subject, the apparatus comprising: a memoryin communication with a processor, the memory storing a computer programrecorded therein, the computer program being executable by the processorto cause the apparatus at least to: determine a likelihood of how theevent is similar to at least one event of a list of events relating tothe unauthorized subject, each event of the list of events comprisingdata identifying the unauthorized subject, the determination oflikelihood being based on the data identifying the unauthorized subject;and determine the event to relate to the unauthorized subject inresponse to the determination of the likelihood.
 9. The apparatus ofclaim 8, wherein the computer program is executed by the processor tocause the apparatus further to: determine a likelihood of how the eventis similar to at least one of: (i) at least one event of a list ofevents relating to the unauthorized subject and (ii) at least one of alist of events relating to an authorized subject, each event of the listof events relating to the authorized subject comprising data identifyingthe unauthorized subject, the determination of likelihood being based onthe data identifying the unauthorized subject.
 10. The apparatus ofclaim 9, wherein the computer program is executed by the processor tocause the apparatus further to: determine if an event of the list ofevents relating to the authorized subject has a matching score higherthan a second matching threshold, the second matching thresholdreferring to a minimum matching score for an event to be determined asbeing related to the authorized subject, the matching score higher thanthe second matching threshold indicating the event of the list of eventsrelates to the authorized subject, wherein the second matching thresholdis higher than a first matching threshold.
 11. The apparatus of claim10, wherein the computer program is executed by the processor to causethe apparatus further to: calculate a matching score of the event basedon data identifying the unauthorized subject; and determine if thematching score of the event is higher than the first matching thresholdand lower than the second matching threshold, the matching score higherthan the first matching threshold and lower than the second matchingthreshold indicating that the data identifying the unauthorized subjecthas a moderate degree of correlation with the data identifying theauthorized subject.
 12. The apparatus of claim 8, wherein the computerprogram is executed by the processor to cause the apparatus further to:determine the event to relate to the authorized subject in response todetermining the likelihood of how the event is similar to the at leastone event of the list of event relating to the authorized subject higherthan the likelihood of how the event is similar to the at least oneevent of the list of event relating to the unauthorized subject.
 13. Theapparatus in claim 9, wherein the computer program is executed by theprocessor to cause the apparatus further to: update the data identifyingthe authorized subject to comprise the data identifying the unauthorizedsubject of each event of the list of events relating the authorizedsubject.
 14. The apparatus of claim 8, wherein the computer program isexecuted by the processor to cause the apparatus further to: receive aninput, the input being at least one image captured by at least one imagecapturing device during a specific time period, wherein the detection ofthe event and the list of events comprising the data identifying theunauthorized subject is based on the received input.
 15. A system fordetermining if an event relates to an unauthorized subject, the systemcomprising: the apparatus as claimed in claim 8 and at least one imagecapturing device.
 16. A non-transitory computer readable medium storinga program for causing a computer to execute: determining a likelihood ofhow the event is similar to at least one of: (i) at least one event of alist of events relating to the unauthorized subject and (ii) at leastone of a list of events relating to an authorized subject, each event ofthe list of events comprising data identifying the unauthorized subject,the determination of likelihood being based on the data identifying theunauthorized subject; and determining the event to relate to theunauthorized subject in response to the determination of the likelihood.